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INTRODUCTION 



1 . 



Hearn and Lawphongpan i ch (1987) studied the ascent nature of 
generalized linear programming (GLP) which is also known as the 
dual cutting plane or primal column generation technique. In 
particular, they examined the properties of the direction, daLP , 
defined as the difference of two successive dual iterates 
generated by GLP with respect to the Lagrangian dual function, L. 
They showed that doLP is an ascent direction for L at points 
where L is differentiable. At nondi f f erent iable points, the 
column entering the master problem is not unique and an arbitrary 
choice can make do L p a nonascent direction. To obtain an 
additional insight on the ascent nature of doLP , they also showed 
that daLP is a solution to a direction finding problem for L. 

In this paper, we introduce a new class of direction finding 
problems and showed that it includes the one which produces d G LP. 
This class of problems contains direction finding problems which 
always produce ascent directions as well as those that do not, 
e.g., the doLP direction finding problem. In order to improve the 
rate of convergence of GLP, we replace the daLP direction finding 
problem with another from this new class. This new direction 
finding problem which we describe below guarantees to produce 
ascent directions and is similar to the one used by Demayanov for 
the minimax problem. Moreover, a line search step is also 
included in this modification of GLP. Hearn and Lawphongpan i ch 
(1987) indicated via a numerical example that a line search step 



reduces the number of iterations for GLP. 



For the remainder of the paper, we state GLP algorithm in 
both the cutting plane and the column generation form and define 
the associated Lagrangian dual problem in Section 2. Then, 
Section 3 describes the new class of direction finding problems. 
Finally, we present the modification of GLP and its convergence 
analysis in Section 4. 



2 . 



PRELIMINARIES 



Consider the primal problem: 

PI: f*=min f ( x) 

x 

s . t . g ( x) <i b , 
x e X , 

where f is a continuous real-valued function, g is a continuous 
function from R n to R“ , X is a nonempty compact subset of R" , and 
b is a vector in R m . The Lagrangian dual of (PI) is: 

Dl: L* = maximize { L(u) : u £ 0 , u e R m } 

where L(u) = minimum (f(x) + u[g(x)-b] : x e X) , and xy denotes 

the usual dot product between vectors x and y. 

When L(u) is relatively easy to evaluate, the pair of 
problems, (PI) and (Dl) , can be addressed by GLP . Below we state 
the algorithm in the dual cutting plane form (Zangwill, 1969). 

The Dual Cutting Plane Algorithm 

Step 0 . Find a point x 0 e X such that g(x 0 ) < b. Let k = 1, and 

go to Step 1. 

Step 1 . Solve the k-th master problem: 

Ml : max w 

( w , u) 

s.t. w <: f (xi) + u[g(xi)-b] for i = 0 k-1 



u £ 0 



Let (w k)Uk ) be an optimal solution and go to Step 2. 



Step 2 . Solve the k-th subproblem: 

SI: min { f(x) + u k [g(x)-b] : xeX) 

Let x k be an optimal point, and let L(u k ) = f (x k ) + 
u k [ g (x k ) -b ] . If w k = L(u k ) , u k is an optimal dual 
solution. Otherwise, if w k > L(u k ) , then replace k by 
k + 1 , and go to Step 1 . 

The k-th master problem (Ml) is a linear programming problem 
with the following dual: 



k- 1 

M2 : min 2 n t f (xj 

n i = 0 

k-1 

s.t. 2 nig(xt) £ b 
i = 0 

k- 1 

2 Hi = 1 , 

i = 0 

n i^O for i = 0 k-1 



When (M2) replaces (Ml) in Step 1, the resulting algorithm 
is generally known as Dantz i g- Wo 1 f e decomposition (1960, 1961) , 

column generation or GLP . Geoffrion (1970) also classifies the 
algorithm in the cutting plane form as the strategy of outer 
linearization and relaxation, or, in the column generating form 
as the strategy of inner linearization and restriction. 



3. 



A CLASS OF DIRECTION FINDING PROBLEMS 



Consider now a feasible direction scheme for solving (Dl) . 

At a given feasible point u* and a parameter a, one possible 
direction finding problem is 

DF1: Max t g (x*) -b ] ( v-u*) 

v 

s . t . a £ L ( v) , 
v £ 0 , 

where x* is a solution of (SI), so that [g(xic)-b] is a 
subgradient of L at u* . The first constraint of (DF1) requires 
that v be in the level set of L as defined by a. If a > L(u*) , 
any feasible solution v to (DF1) would produce an ascend 
direction of the form v-ujc, and the best value for a is L* . 

Figure 3.1 illustrates an instance of (DF1) in which cx is set to 
L(ujc) (= 7.0 ) and the optimal solution of the resulting problem 
is denoted by v* . Note that the direction Vic-Ujc is an ascent 
direction for any choice of subgradients. 

In practice, L(v) must be approximated since it is not 
always available in closed form. Each different approximation 
would yield a different direction finding problem. Similarly, the 
different choices for a and xic would also yield different 
direction finding problems. To ensure that vic-u* is an ascent 
direction, the approximating function along with the value for a 
and the vector Xic must be carefully chosen. Below, we examine two 
different direction finding problems derived from (DF1). 




Figure 3.1 : An illustration of direction finding problem (DF1) 



First, we consider the direction finding problem associated 
with GLP in the cutting plane form. From the master problem (Ml), 
it is clear that the approximation of L is 

Lai(v) = min { f(Xi) + v[g(x±)-b]: i=0,...,k-l }. 

Then, Hearn and Lawphongpan i ch ( 1987) showed that the 

solution to the (k+l)-st master problem, solves the following 
problem derived from (DF1) : 

DF2: max [ g (Xk) -b ] ( v-u*) 

v 

S . t . Wjc -► i ^ Lai ( v) , 
v ^ 0 , 

where w k -ni , the optimal objective function value of the (k+l)-st 
master problem, replaces a. Define X(ujc) as the set of solutions 
to (SI). When X(u*) is a singleton, L is differentiable at u* , 
and [g(xjc)-b] is the only choice of ' subgrad i ent ’ for the 
objective function of (DF2) . However, when X(uk) is not a 
singleton, L is nond i f f er en t iabl e and the choice of objective 
functions becomes infinite. Moreover, an arbitrarily chosen 
subgradient could result in (DF2) generating a nonascent 
direction as shown in Hearn and Lawphongpan i ch (1987) . Below, we 
consider a second direction finding problem in which [g(x*)-b] is 
replaced with a set that includes the subdifferential of L at u* . 

Assume that X can be represented by a finite discrete set 
(y i,...,y T ). For example, when (PI) is a bounded integer program, 
then X is itself a finite set of vectors with integral elements. 



Then, X(uic) is a finite discrete set. For the discussion below, 
define for any u i 0 

Xc(u) = { yj : f (y j ) +u (g (yj ) -b) £ L(u) + e , yj e X } , 

that is, X«-(u) is the set of all e -approximate solutions of (SI) . 
When e = 0, Xo(u) is simply the set of solutions to (SI) which is 
previously denoted as X(u) . Since L is concave, the e- 
subdi f f erential of L is well defined and can be written as 
follows (see, e.g., Clark, 1975; Kiwiel, 1985; Lemarechel, 1980; 
and Zowe , 1987 ) : 

6 c L(u) = { h: L(v) £ L(u) + h(v-u) + e, V v £ 0 } 

Similarly, when e = 0, 8oL(u) = <5L(u) , the subdifferential of L 
at u, and 8L(u) is also equivalent to conv{ [g(yj)-b]: yj e 

X(u)}, where conv(') denotes the convex hull of a set. Below, 
Theorem 3.1 relates X«r(u) to the E-subdifferential of L(u). 

Theorem 3.1 ; Conv{ [g(yj)-b]: yj e X c (u) ) is a subset of 6 c L(u) . 

Proof : For every v £ 0 and for each yj e Xt(u) 

L(u) + [ g (y j ) -b ] ( v-u) = L(u) - [g(yj)-b]u + [g(yj)-b]v 

£ f (yj ) - e + [g (yj ) -b]v 

£ L ( v ) - e , or 

L(u) + [ g (y j ) -b ] ( v-u) + e <£ L(v), (3.1) 

where the first inequality follows from the definition of X c (u) 
and the second inequality from the definition of L. Since 



equation (3.1) holds for all yj e X c (u), it must hold for all 
convex combinations of yj e Xt(u), and the theorem follows. Jj 

Given the above definitions, (DF1) can be modified as 
foil ows : 

DF3 : max min { [ g (yj ) -b ] ( v-u* ) : yj e X«(u k ) ) 

s . t . L (ui<) £ Las ( v) 

v i 0 , 

where 

La 2 = min { f (x) +-v[g(x)-b]: x e M* -1 U (xo) }, and 

k- 1 

M*" 1 = any subset of U X c (ui) . 

i = 1 

If X(uic) replaces X«r(ujc) in the objective function of (DF3) , the 
’min* part in the objective is the expression for the directional 
derivative of L at u* in the direction (v-u*) . Note that (DF3) 
can be written in the cutting plane format as follows: 

DF 4 : max w 

s.t. w £ [ g (y j ) -b ] ( v-u*) V yj e X e (uk) 

L(uic) £ f (x) + v[g(x)-b] V x e M* -1 U (xo) 

v ^ 0 . 

Under the assumption that X is a finite discrete set, L(u) 
can be equivalently written as a minimum of a finite number of 
linear functions. Then, (DF3) and (DF4) resemble the direction 
finding problem of an algorithm proposed by Demyanov (see, 
Demyanov and Malozemov, 1974, and Lemarechel, 1980) in that they 
require a full knowledge of the e-subdifferential. However, 



Demyanov's direction finding problem is nonlinear whereas (DF4) 
is 1 inear . 

As stated in (DF3) , can be any subset of the union of 

the previously calculated E-subdif ferentials . Theoretically, M*" 1 
can be an empty set and the algorithm to be presented in the next 
section would still converge. However, taking M*' 1 as an empty 
set means that L(v) is approximated by only one hyperplane 
defined by Xo. In practice, this may not yield good directions. 
Thus, the choice of the set M* -1 should allow a good 
approximation of L(v) and, in turn, its contours in the 



neighborhood of Uk. 



4 . 



An Ascent Algorithm and its Convergence 



The algorithm below is a modification of the cutting plane 
algorithm in which we replace the master problem (Ml) with the 
direction finding problem (DF3) [or ( equivalently, (DF4)] and add 
a line search step. 



An Ascent Algorithm 

Step 0 . Let Xo e X satify g(xo) < b. Select ui ^ 0 and compute 
X* (ui) . Set M° = 0 and k = 1 . 

Step 1 . (Direction Finding) Solve problem (DF3) (or, 

equivalently, (DF4)) and let w k denote the value of the 
optimal objective function, and v k denote the solution. If 
wjc ^ 0 , stop and uic is an optimal solution. Otherwise, go to 
Step 2 . 

Step 2 . (Line search) Solve 

L(ujc + tu ( vjc-ujc) ) = max { L(ujc + t(vic-uic)): 0 £ t £ 1 } 

t 

and set uic+i = u* + tjc(vu-uic) . Go to Step 3. 

Step 3 . (Evaluate L(uic-i)) Solve the subproblem (SI) and 

construct the set X«r(uic-i). Set k = k+1, and go to Step 1. 

In Step 0, the point xo satifies the Slater constraint 
qualification, and by construction Xo is included in the 
approximating function L*.^ . This prevents v* from being unbounded 
since v[g (xo) -b] — > -<» as if any component of v goes to <» . For 

the convergence analysis below, it is assumed that the set (u: a 



£ L(u)} is bounded for all a. Thus, u* is bounded for all k as 



we 1 1 . 

The first three theorems justify the direction finding 
problem (DF3) . 

Theorem 4.1 : u* is a solution to (Dl) if and only if u* solves 

(DF3) . 

Proof : For convenience, assume that 

1) X(ufc) = (yi y p J , 

2) Xt(ujc) = {yi y* .y**! y,} , and 

3) M*“ x = (xi , . . . , x<s) . 

where p £ q and q £ T . 

Let u>c be a solution to (Dl) . Then, there exists a (n,a) 
satisfying the following Karush-Kuhn-Tucker (KKT) conditions: 



p m 

2 nj[g(yj)-b] + 2 cxx-e,- 
j = l r = 1 

P 

2 71 i 

j = l 

CKr[U)t)r 

7T and a 



where e»- is the r-th unit vector 
component of the vector z. Given 
define the triplet ( tt ’ ,6’ , a ’ ) as 



Kj - 71 j , j - l,...,p 



= 0 
= 1 

= 0, r = 1 m 

2 0 . 

in R m and [z] r denotes the r-th 
this pair of multipliers, (tt.cx) 
follows: 



"j 


o 

II 


j 


= p+i,. 


■ • • .q 




II 

o 


i 


= i 


■ ,C 


CXr 


— c x r , 


r 


= i 


. , m . 



Then, (n’ ,a’) satisfies the following KKT conditions for (DF3) 
at the point u* : 

q c m 

2 nj[g(yj)-b] + 2 BiCg(xi)-b] + 2 aie r = 0 
j = l i = l r = 1 

q 

2 ni =1 

j = 1 

BI [g (xi ) -b] =0 

OrtUklr = 0 

n ’ , 6 ’ , and a ’ £ 0 . 

Since Uk is feasible to (DF3) , Uk must be optimal to (DF3) as 
well. 



Assume that Uk solves (DF3) . Then, there exists a (n’ ,6’ , a ’ ) 
satisfying the above KKT conditions for (DF3) . Since n’ and 6’ 
are nonnegative and Zjnj = 1 , we can define 



si = ZjHj + 2 1 6 1 



n j /a , 


J = 1 , . 


. . . , P , P+ 1 , . 


• . ,q 


6I/a , 


i = 1 . • 


. . . , s 




C(r / cl , 


r = 1 , 


. . . , m. 





Then, (n'.B’.a') is an optimal set of multipliers at the point 
(z,v) = (L(ui«) , Uj< ) for the following problem which is related to 



(DF4) 



DF5 : max z 

s.t. z £ f(yj) + Cg(yj)-b]v, j = 1 p,p+l,...q 

z £ f (xi) + [g(x±)-b]v, i = 1 , . . . , 3 

v £ 0 , 

that is, (L(uk) ,Uic) solves DF5 . However, note that (Dl) can also 
be wr i t ten as : 

D2: max z 

s.t. z £ L ( v ) 
v 2: 0 . 

Then, (DF5) is a relaxation of (D2) . Since (Ltu*) ,Uk) solves 
(DF5) and is feasible (D2) , it must be optimal to (D2) 

Theorem 4 . 2 : I f u* does not solve (Dl), then 

mint [ g (y j ) -b ] ( vic-uic) : yj e X«.(uic) } > 0, 

where v* solves (DF3) . 

Proof : By Theorem 4.1, u* does not solve (Dl) implies that u* is 

not a solution to (DF3) . However, u* is still feasible to (DF3) , 
so the following inequality must hold: 

mint t g (yj ) -b ] ( vj«-uie) : yj e X^tu*) ) 

= max { mint tg (yj ) -b ] (v-uic) : yj £ Xc(u k ) }} 

v f eas ibl e 

> mint tg (yj ) -b ] (uk-uic) : yj e X c ( ui< ) ) = 0.| 

It should be noted that the above two results also hold when 
La 2 <v) is replaced by the actual Lagrangian function L in (DF3) . 



The following series of lemmas and a theorem demonstrate 
that the ascent algorithm above converges to an optimal solution 
of (Dl) . Furthermore, they are similar to the standard argument 
for establishing convergence of feasible direction algorithms in 
nonlinear programming. 

Lemma 4.3 : There exists an £ > 0 such that if u* — > u* , then 
XdUk) = X(u») , for k sufficiently large. 

Proof : Let X(u*) = (y y p ) . Define 

i) hi(u) = max {f(yj) + u[g(yj)-b]: j = l,...,p) 

ii) hs (u) = min (f(yj) + u[g(yj)-b]: j = p+l,...,T} 

iii) cx = [hi (u* ) +hs (u*) ] /2 

At u* , the following hold: 

A) hi ( u* ) = L ( u* ) = f(yj) + u*[g(yj)-b] , j = l,...,p 

B) hi ( u* ) < a < h 2 (u*) 

C) a = [cx - hi (u*) ] > 0 

From the continuity of hi(u) , h:<(u) , and L(u) , there must exist 

an integer K sufficiently large so that for every k £ K 

D) | hi (uie) - hi (u*) | < a/2 

E) | L (uic) -L (u* ) | < a/4 

F) for j = 1 p 

|f(yj)+u*[g(yj)-b] - L(u*)| 

= | f (y j ) +u* [ g (y j ) -b ] - f (yj ) -u* [g (yj ) -b] | 

< a/4 . 



G) hi ( uic) > cx . 



From (E) and (F) , we have that for j 



1 P 



| f (y j ) +uic [ g (y j ) -b ] -L(uic)| 

£ |f(yj)+u>e[g(yj)-b] - L(u*)| + | L (u* ) - L (u*) | 

£ < 7 / 2 , 

and since L(u*) ^ f(yj) +u*[g (yj ) -b ] , V j, it follows that 

| f (y j ) + ujc [ g (yj ) -b ] -L(u k )| = f (y j ) +u k [ g (yj ) -b ] -L(uic) £ <7/2, or 

f (y j ) + Uj< [ g (y j ) -b ] £ L(u»c)+a/2. 

Thus, for any e between (l/2)a and (3/4)<7, 

f(yj) + Uk[g(yj)-b] £ Mu*) + e, for j = 1 p, (4.1) 

that is, yj is an element of X^(uic) for j = 1 p. Moreover, 

f rom (E) , 

Mu*) £ L(u*) + <7/4 

L(uj<) + e £ L(u*) + <7/4 + e 

£ L (u# ) + <7 

= a 

< f(yj)+u*[g(yj)-b], for j = p+1 T. (4.2) 

where the third inequality follows the above selection of e, the 
equality from (C) , and the last inequality from (G) and the 
definition of h 2 (u) . This means that yj does not belong to 
Xe(uk) , for j = p + 1 , . . . , T . Therefore, X c (u*) = X(u*). g 



Lemma 4.4 : For a given direction d, 



mini g'd: g" e conv ( [ g (y j ) -b ] : j = l t ...,p) } 

= mini i g (y j ) - b ]d : j = 1 p } 

Proo f : The result follows from the fact that the problem on the 
left hand side can be stated as a linear program 

P 

min 2 ( tg (yj ) -b]d) nj 

j = l 

P 

s . t . 2 n j = 1 

j = l 

^ 0 , j = 1 , . . . , p , 

which always yields an extreme point solution. 

Lemma 4.5 : If Xc(u k ) = (yi, . . . ,y p ) and mint [g(yj)-b]d: j = 

1 p } > 0, then there exists a t > 0 such that 

Liuic + od) 2 L(Uk)+(?min( [g(yj)-b]d: j = 1 p } 

for all 0 £ a £ -t. 

Proof : Assume without loss of generality that 

0 < [g(yi)-b]d = mini [g(yj)-b]d: j = l,...,p } 



Define 



T 



min 



i f (y j ) +u[g (y j ) -b ] - L(uk) } 

[ g (y 1 ) ~ g (yj ) ]d 



for 
. g (y 1 ) d 



all j such that 
> g(yj)d & j t Xt(Uk) 



and observe that -t > 0 since 



f (y j ) + utg(yj)-b] > L(uk) 



for j t Xc(Uk) , 



and [g (y i ) -g (y j ) ] d > 0 by construction. Thus, for any 0 £ ct £ t 

and every j % X®(ujc) and g(yi)d > g(yj)d 



(f (y 

CT - 

a[g(yi)-b - g(yj)+b]d £ f (yj 
L(u»c) + a[g(yx)-b]d ^ f(yj) 

However, for j t X«-(uic) and 0 < 

L (uic) < f (y j ) 
L(uic) + <r[g(yx)-b]d < f(yj) 

and f or j e X* (uic) 

L (uic ) i f (yj ) 
L(ujc) + a[g(yi)-b]d £ f ( y j ) 



) + u>c[g(yj)-b] - L(uic) 

[g (y i) - g (yj ) ] d 
+ ujc[g(yj)-b] - L(uic) 

+ (uic + ad ) [ g ( u j ) -b ] (4.3) 

[g(yi)~b]d i [g(yj)-b]d 

+ uic [ g (yj ) -b ] 

+ (ujc + ad) [g (yj ) -b] (4.4) 

+ uic [ g (y j ) -b ] 

+ (Uk + ad) [g (yj ) -b ] . (4.5) 



Combining (4.3) , (4.4) and (4.5) , we have that for 0 £ a £ t 



L(ujc) + a[g(yi)-b]d £ mint f ( y j ) + ( u* + ad ) [ g (y j ) - b ] : j = l T) 

= L (uic + ad) . | 

Lemma 4.6 : Assume that e is chosen as in Lemma 4.3 and the 

algorithm generates a sequence {u>c} . Then, there cannot be a 
subsequence tu*} , k e ft, with the following properties: 



i) 


Uk — > 


u« , 


k e 


n. 


ii) 


Vk > 


Vo» , 


k e 


ft , and 


iii) 


mint g 


^8 

P 

8 

> 

• 


: g" 


e conv ( [g (y ) -b ] : y e X(u-)) } > 0, 



where Q is a subset of {1,2,3,...}. 



Proof : Assume that X(u») = (1 , . . . ,p) . By (iii) and Lemma 4.4, 

min{ g”(v»-u»): g" s conv ( [ g ( y ) - b ] : y s X(u»)) } 

= min { t g (y j ) -b 3 ( v»-u») : j = 1 p } = B > 0. 

From Lemma 4.3, there must exist a K 1 sufficiently large such 
that X^(uk) = {l,...,p> for some k k K 1 and k s ft. Since the set 
(l,...p) is finite, there must exist a index j* such that 

j* = arg mint [ g (y j ) -b ] ( v*-uk) : j = l,...,p } (4.6) 

infinitely often. Define ft 1 to be the subset of ft for which j* is 
the index which yields the minimum value for the right hand side 
of (4.6) . For convenience, we assume that j* = 1. Then, we have 

that 

Lim [ g (y i ) -b ] ( vie-uic ) = [g (y i) -b] (v»-u») = 6 > 0, 

k £ ft 1 

and it follows that there exists K 2 £ K 1 such that 

tg(yi) -b] (vie-uie) > B/2 for k £ K 2 and k e ft 1 . (4.7) 

Thus, at u>c the direction (vic-u*) is an ascent direction. 
Moreover, since uic-i maximizes L(u) along the direction (vk-u*) , 

L(uie-i) £ L(uie + ct(v*-uic)) for 0 < <7 < t and k e ft 1 

£ L(uie) + cr [ g (y i ) -b ] ( Vk-Uic) for 0 < a < t and k e ft 1 

£ L(uic) + (ctB) / 2 for 0 < ct < t and k e ft 1 , (4.8) 

where t is as defined in Lemma 4.5 and the last two inequalities 
follow from Lemma 4.5 and equ. (4.7) . Letting k approach infinity 
on the subsequence ft 1 , (4.8) yields 



L(u«) £ L(u«) + (cr&) / 2 



which is a contradiction since both a and & are positive.] 

Theorem 4.7 : Assume that e is chosen as in Lemma 4.3. If the 

algorithm generates a sequence {u*} , then it must converge 
to an optimal solution of (Dl). 

Proof : Assume that the algorithm generates an infinite sequence 

of points and none of which is a solution to (Dl) . Since tu*) 
lies in a bounded region, there must exist a convergent 
subsequence, i.e., u* — > u« , for k e ft, where ft is a subset of 
(1,2,3,...) and u«> does not solve (Dl). 

Assume that X(u») = (l,...,p). By Lemma 4.3, X c ( u* ) = X(u«) 

for k sufficiently large, i.e., k £ Ki, thus X«r(u«) = X(u«). 

Since u« is not a solution to (Dl) , 

0 < &co = max mint C g (y j ) -b ] ( z-u») : j = 1 p } 

z j 

s.t. L(u«>) £L(z) 

z £ 0 . 

For k £ Ki , define 

DF6 : &ic = max min{ [ g (y j ) -b ] ( z-u*) : j = l,...,p } 

z j 

s.t. L(uic) ^L(z) 

z £ 0 . 

Note that varies continuously with u k . Thus, for k e 



ft and k £ K i . 



Now, let Vk be a solution to (DF3) defined at u* . Then, by 
construction, v* is bounded and there must exist a subset, ft 1 , of 

ft such that v* > v«, on ft 1 . Moreover, since (DF3) is a 

relaxation of (DF6) , 

£ min{ [ g (yj ) -b ] ( Vjc-Ujc) : j = l,...,p }, (4.9) 

for k £ Ki and k e ft 1 . From the finiteness of of the set 
(l,... f p), there must exist an integer j* such that 

j* = arg mint [ g (y j ) -b ] ( vjc-u*) : j = l,...,p } (4.10) 

infinitely often. Let ft 2 be a subset of ft 1 for which j* is the 
index which minimizes the right hand side of (4.10) and for 
convenience assume that j* = 1. Then, combining (4.9) and (4.10) 

gives 

[ g (y i ) -b ] ( Vk-Uk) ^ Bic for k ^ Ki and ft 2 . 

Taking the limit on both side with respect to ft 2 , we have that 
lim [g (y i ) -b] (vic-uic) = [g(yi) -b] (v»-u«) ^ > 0 

which contradicts Lemma 4.6. g 

Therefore, if the algorithm terminates after a finite 
number of iterations, Theorem 4.1 and 4.2 guarantee that u* 
solves (Dl). Otherwise, the algorithm generates an infinite 
sequence which, by Theorem 4.7, converges to an optimal solution 
of (Dl) . Also, it is interesting to note that although the 
algorithm uses e-subdifferential in calculating its ascent 
directions an exact optimal solution can be obtained by choosing 



£ correctly. In general, one expects algorithms using an e- 
subdifferential to produce e-optimal solutions, possibly in a 
finite number of iterations. We consider this type of algorithm 
in a separate study to appear later. 
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